Imagine if everyone with an Amazon Echo in their home had to assume it’s been unlocking their doors and letting a thief inside for the past 6 months. How would you figure out what’s missing if they copied your keys, your hard drives, or whatever is in your file cabinet, poisoned your water supply?
That’s the position thousands of organizations affected by the SolarWinds SUNBURST supply chain attack are in right now. They’re frantically hunting for signs of compromise or setting fire to their infrastructure to be (mostly) certain the Russian APT is purged.
On December 13, FireEye shared valuable details on the breach about how threat actors compromised SolarWinds Orion software update distribution mechanism to spread malicious code to organizations using the software.
The CISA issued an emergency alert on December 17th, asking all companies using SolarWinds to upgrade or even disconnect their SolarWinds Orion from the network. Varonis security teams have since seen a spike in forensics investigations related to these findings and have identified several ongoing attacks that will be covered in this update.
Though most of the coverage to date has centered around fallout from compromised versions of SolarWinds’ Orion solution, according to CISA, there is evidence of additional intrusion vectors associated with this campaign.
Supply chain attacks are difficult to defend against, and this one is especially bad
In a supply chain attack, an attacker goes after a trusted vendor or product instead of attacking their targets directly. In this case, the attackers introduced a pre-crafted backdoor into a trusted software product (SolarWinds Orion) that was delivered automatically to thousands of customers, disguised as a normal update.
The bad news didn’t stop here – the threat actors were stealthy enough to stay hidden for months. They had time to plant backdoors and access a lot of systems and data. Organizations are now examining everything they can, starting with the systems and accounts associated with SolarWinds and working their way outwards.
Triage Steps
Whether you’re a Varonis customer or not, step one is to check for a vulnerable version of SolarWinds. SolarWinds has identified the affected versions and, as of 12/16/20, has released updates and hotfixes to replace the compromised components.
If you’re vulnerable, here’s what you can do next:
1) Check for connections to avsvmcloud[.]com and other domains associated with the attack
Examine DNS and proxy connections to activity on the avsvmcloud[.]com domains, as well as other domains identified and associated with SolarWinds SUNBURST command and control (C2).
Varonis customers can easily search for alerts and events related to activity on these domains using the Edge dashboard and threat hunting queries.
2) Check for unusual activity by accounts and systems associated with SolarWinds
Varonis customers have detected unusual activity by the service accounts associated with SolarWinds, including unusual file activity and connections to end-user computers (“personal devices”). Unusual connections, domain changes, or file system activity by SolarWinds service accounts should be investigated, especially related to sensitive data or systems.
Here’s an example of a Varonis alert detecting a SolarWinds service account performing abnormal file system actions that it doesn’t typically perform:
3) Check for unusual resource access and changes made by other service accounts (not just those associated with SolarWinds).
Review alerts and activity for unusual authentication and access behavior, paying extra attention to alerts and activity from service accounts and service principals in hybrid environments (AAD).
If you’d like our IR team’s help investigating, please reach out to your sales team or request a DatAlert office hours session here.
4) Varonis customers will get the latest Sunburst IOC’s thru Live update (if enabled) and can enable an additional rule to accelerate detection.
What We Know About This APT Campaign (Technical Analysis)
This attack was performed without weaponizing a zero-day vulnerability (that we know of). The prevailing theory, not yet confirmed by SolarWinds, is that the attackers used exposed FTP server credentials found on GitHub in 2018 to gain access to the company’s software update infrastructure.
The attacker was able to modify the software updates and add a malicious backdoor to one of SolarWinds Orion plugin DLLs called SolarWinds.Orion.Core.BusinessLayer.dll.
The attackers signed their malicious version of the DLL with the SolarWinds private key. The certificate was issued by Symantec.
We assume the attacker was able to sign the DLL in one of two ways:
- The attacker got a foothold in the development process, added the backdoor, and let SolarWinds sign it as part of the deployment process.
- The attacker stole the certificate’s private key, signed the DLL themselves, and replaced the official DLL with their malicious version. This is less likely.
Any organization that uses SolarWinds and receives updates from their servers would have downloaded and executed the malicious DLL. Because the DLL was signed and delivered through the official SolarWinds update servers, it would be extremely difficult to detect the malicious content.
Analyzing the SolarWinds SUNBURST Backdoor (BusinessLayer.dll)
When we look inside the malicious DLL, we see that the attacker had stealth in mind. They went to a lot of effort to use code that would blend in with the rest of Orion’s source code, using well-written arguments and generic, unsuspicious class and method names like “initialize,” and “job.”
The SolarWinds SUNBURST backdoor executes in several stages:
- Ticking time bomb
- The SolarWinds SUNBURST backdoor waits 12-14 days before sending its first beacon to the C2 server. This makes it much harder to detect and to relate the attack to the malicious update.
- Information gathering.
- The SolarWinds SUNBURST backdoor sends some basic information back to the C2 server (username, IP address, OS version) to determine if the machine is worth exploring.
- Communication
- The backdoor uses a custom domain generation algorithm (DGA) to determine its Command and Control (C2) IP address. When communicating with the C2 server, the backdoor mimics legitimate SolarWinds OIP (Orion Improvement Program) communication.
- Evasion
- The code scans the machine for anti-virus and forensics tools that could detect its presence.
- Dropper
- If the attacker thinks that the network is worth exploring, they will use the backdoor to download a small dropper known as TEARDROP, which will then download additional tools for post-intrusion activities.
How the C2 Works
The initial C2 communication sends encoded DNS packets with info about the victim’s device and environment.
The thing that makes SUNBURST’s initial C2 so interesting is that the IP Address in the DNS response determines the malware’s next move. Based on the IP address range in the response, the SUNBURST process either kills itself, waits, or runs another function, like disabling antivirus or executing an additional backdoor.
Let’s look and see how this initial DNS-based C2 works:
1) Once the DLL is loaded, it performs a series of checks to ensure that it’s running on an enterprise network, and not on an isolated machine.
2) The attackers used what our researchers refer to as a “subdomain generation algorithm” to build a unique subdomain for each victim. The algorithm gathers local device information and encodes it into a small text string.
3) Then, it builds a portion of the fully qualified C2 domain name (FQDN) using two hard-coded strings and a string array containing four values:
The decoded values are:
Domain1 = ‘avsvmcloud[.]com’
Domain2 = ‘appsync-api’
Domain3 = [‘eu-west-1’, ‘us-west-2’, ‘us-east-1’, ‘us-east-2’]
The SUNBURST code then concatenates the values together using the GetStatus function:
4) Next, it prepends the victim’s information to the above to form the FQDN and issues a request to resolve the FQDN. At this point, the data will reach the authoritative domain name server controlled by the attacker.
There are four functions used to get and prepend the victim’s information:
‘GetCurrentString’ and ‘GetPreviousString’ are used to provide a unique GUID and the hostname/domain of the device.
‘GetNextString’ and ‘GetNextStringEx’ are used to provide the running processes and their status along with the GUID.
By providing this info in each DNS request, the C2 server can make educated decisions about how to respond.
Here are some examples of FQDNs queried by SUNBURST: 
(Source)
Analysts at Prevasio have decoded some of the subdomains found in DNS logs to identify the victims.
5) As mentioned above, the IP address in the DNS response determines the response within the SUNBURST code. The “IPAddressHelper” class contains hardcoded IP ranges that will map the IPs in the response to an IP Adress Family:
Encoded IP ranges classified to AddressFamilyEX types
Depending on which IP address is returned by the attacker, SUNBURST will execute methods to enumerate current processes, invoke its HTTP backdoor, and more.
6) Instead of responding with an IP address, the DNS server may redirect the victim by responding with CNAME that points to a secondary C2 domain that is used for further C2 and exfiltration:
Publicly published secondary C2 domains
7) To avoid detection, SUNBURST makes its DNS requests slowly over time, using a random delay for each victim/domain, in some cases waiting up to 120 minutes between requests.
8) SUNBURST also sends HTTP requests to the C2 domain using a pseudo-random, benign-looking URI to invoke commands, passing payload within the HTTP body in JSON format. An example URI:
hxxps://3mu76044hgf7shjf[.]appsync-api[.]eu-west-[.]avsvmcloud[.]com /swip/upd/Orion[.]Wireless[.]xml
The response can trigger several commands supported by the SUNBURST backdoor used to read/write files, restart the device, etc.:
Detecting the Malicious DLL
Unfortunately, SolarWinds advises customers to exclude their software from anti-virus and EDR scans to avoid false positives. Many security teams apply these types of exclusions on their own to reduce noise.
Once this SolarWinds SUNBURST backdoor is in place, the attacker used a dropper called TEARDROP which reads from a fake image file named “gracious_truth.jpg” to decode an embedded Cobalt Strike Beacon payload, customized by the threat actors.
From here, the post-intrusion activities begin—lateral movement, privilege escalation, stealing data, and establishing further persistence.
FireEye and CISA have done a fantastic job documenting the IOCs related to the stealthy post-intrusion activity observed. We cannot assume that every victim’s post-intrusion forensics will look the same. In fact, it’s safe to say that the threat actors are more likely to use highly customized, human-operated campaigns to steal specific information from each victim—e.g., FireEye’s red team tools.
Supply chain attacks like these make a strong case for adopting a Zero-trust model and behavior-based detections. The attackers were able to make their malicious version of the SolarWinds Orion DLL look like a normal version of the software. It was virtually impossible to detect because everything looked official. But as they begin to move through a network, accessing new accounts, and touching data, the actors, as sophisticated as they are, don’t know how to precisely mimic the normal behavior of all the users and devices they’re operating—and that opens a new window of opportunity for detection.
CISA calls out the importance of behavioral modeling in its alert about this APT:
“Since valid, but unauthorized, security tokens and accounts are utilized, detecting this activity will require the maturity to identify actions that are outside of a user’s normal duties. For example, it is unlikely that an account associated with the HR department would need to access the cyber threat intelligence database.”
